Ambient assisted living for frail people through human activity recognition: state-of-the-art, challenges and future directions.

Ambient Assisted Living is a concept that focuses on using technology to support and enhance the quality of life and well-being of frail or elderly individuals in both indoor and outdoor environments. It aims at empowering individuals to maintain their independence and autonomy while ensuring their safety and providing assistance when needed. Human Activity Recognition is widely regarded as the most popular methodology within the field of Ambient Assisted Living. Human Activity Recognition involves automatically detecting and classifying the activities performed by individuals using sensor-based systems. Researchers have employed various methodologies, utilizing wearable and/or non-wearable sensors, and employing algorithms ranging from simple threshold-based techniques to more advanced deep learning approaches. In this review, literature from the past decade is critically examined, specifically exploring the technological aspects of Human Activity Recognition in Ambient Assisted Living. An exhaustive analysis of the methodologies adopted, highlighting their strengths and weaknesses is provided. Finally, challenges encountered in the field of Human Activity Recognition for Ambient Assisted Living are thoroughly discussed. These challenges encompass issues related to data collection, model training, real-time performance, generalizability, and user acceptance. Miniaturization, unobtrusiveness, energy harvesting and communication efficiency will be the crucial factors for new wearable solutions.

Recurrent Network Solutions for Human Posture Recognition Based on Kinect Skeletal Data.

Ambient Assisted Living (AAL) systems are designed to provide unobtrusive and user-friendly support in daily life and can be used for monitoring frail people based on various types of sensors, including wearables and cameras. Although cameras can be perceived as intrusive in terms of privacy, low-cost RGB-D devices (i.e., Kinect V2) that extract skeletal data can partially overcome these limits. In addition, deep learning-based algorithms, such as Recurrent Neural Networks (RNNs), can be trained on skeletal tracking data to automatically identify different human postures in the AAL domain. In this study, we investigate the performance of two RNN models (2BLSTM and 3BGRU) in identifying daily living postures and potentially dangerous situations in a home monitoring system, based on 3D skeletal data acquired with Kinect V2. We tested the RNN models with two different feature sets: one consisting of eight human-crafted kinematic features selected by a genetic algorithm, and another consisting of 52 ego-centric 3D coordinates of each considered skeleton joint, plus the subject's distance from the Kinect V2. To improve the generalization ability of the 3BGRU model, we also applied a data augmentation method to balance the training dataset. With this last solution we reached an accuracy of 88%, the best we achieved so far.

Training-Induced Muscle Fatigue with a Powered Lower-Limb Exoskeleton: A Preliminary Study on Healthy Subjects.

Powered lower-limb exoskeletons represent a promising technology for helping the upright stance and gait of people with lower-body paralysis or severe paresis from spinal cord injury. The powered lower-limb exoskeleton assistance can reduce the development of lower-limb muscular fatigue as a risk factor for spasticity. Therefore, measuring powered lower-limb exoskeleton training-induced fatigue is relevant to guiding and improving such technology's development. In this preliminary study, thirty healthy subjects (age 23.2 ± 2.7 years) performed three motor tasks: (i) walking overground (WO), (ii) treadmill walking (WT), (iii) standing and sitting (STS) in three separate exoskeleton-based training sessions of 60 min each. The changes in the production of lower-limb maximal voluntary isometric contraction (MVIC) were assessed for knee and ankle dorsiflexion and extension before and after the three exoskeleton-based trained motor tasks. The MVIC forces decreased significantly after the three trained motor tasks except for the ankle dorsiflexion. However, no significant interaction was found between time (before-, and after-training) and the training sessions except for the knee flexion, where significant fatigue was induced by WO and WT trained motor tasks. The results of this study pose the basis to generate data useful for a better approach to the exoskeleton-based training. The STS task leads to a lower level of muscular fatigue, especially for the knee flexor muscles.

Neural Networks for Automatic Posture Recognition in Ambient-Assisted Living.

Human Action Recognition (HAR) is a rapidly evolving field impacting numerous domains, among which is Ambient Assisted Living (AAL). In such a context, the aim of HAR is meeting the needs of frail individuals, whether elderly and/or disabled and promoting autonomous, safe and secure living. To this goal, we propose a monitoring system detecting dangerous situations by classifying human postures through Artificial Intelligence (AI) solutions. The developed algorithm works on a set of features computed from the skeleton data provided by four Kinect One systems simultaneously recording the scene from different angles and identifying the posture of the subject in an ecological context within each recorded frame. Here, we compare the recognition abilities of Multi-Layer Perceptron (MLP) and Long-Short Term Memory (LSTM) Sequence networks. Starting from the set of previously selected features we performed a further feature selection based on an SVM algorithm for the optimization of the MLP network and used a genetic algorithm for selecting the features for the LSTM sequence model. We then optimized the architecture and hyperparameters of both models before comparing their performances. The best MLP model (3 hidden layers and a Softmax output layer) achieved 78.4%, while the best LSTM (2 bidirectional LSTM layers, 2 dropout and a fully connected layer) reached 85.7%. The analysis of the performances on individual classes highlights the better suitability of the LSTM approach.

Skeleton data pre-processing for human pose recognition using Neural Network.

Automatic monitoring of daily living activities can greatly improve the possibility of living autonomously for frail individuals. Pose recognition based on skeleton tracking data is promising for identifying dangerous situations and trigger external intervention or other alarms, while avoiding privacy issues and the need for patient compliance. Here we present the benefits of pre-processing Kinect-recorded skeleton data to limit the several errors produced by the system when the subject is not in ideal tracking conditions. The accuracy of our two hidden layers MLP classifier improved from about 82% to over 92% in recognizing actors in four different poses: standing, sitting, lying and dangerous sitting.

Automatic Pose Recognition for Monitoring Dangerous Situations in Ambient-Assisted Living.

Continuous monitoring of frail individuals for detecting dangerous situations during their daily living at home can be a powerful tool toward their inclusion in the society by allowing living independently while safely. To this goal we developed a pose recognition system tailored to disabled students living in college dorms and based on skeleton tracking through four Kinect One devices independently recording the inhabitant with different viewpoints, while preserving the individual's privacy. The system is intended to classify each data frame and provide the classification result to a further decision-making algorithm, which may trigger an alarm based on the classified pose and the location of the subject with respect to the furniture in the room. An extensive dataset was recorded on 12 individuals moving in a mockup room and undertaking four poses to be recognized: standing, sitting, lying down, and "dangerous sitting." The latter consists of the subject slumped in a chair with his/her head lying forward or backward as if unconscious. Each skeleton frame was labeled and represented using 10 discriminative features: three skeletal joint vertical coordinates and seven relative and absolute angles describing articular joint positions and body segment orientation. In order to classify the pose of the subject in each skeleton frame we built a two hidden layers multi-layer perceptron neural network with a "SoftMax" output layer, which we trained on the data from 10 of the 12 subjects (495,728 frames), with the data from the two remaining subjects representing the test set (106,802 frames). The system achieved very promising results, with an average accuracy of 83.9% (ranging 82.7 and 94.3% in each of the four classes). Our work proves the usefulness of human pose recognition based on machine learning in the field of safety monitoring in assisted living conditions.

Temporal features of postural adaptation strategy to prolonged and repeatable balance perturbation.

Aim of this study was to get insight into the features of the postural adaptation process, occurring during a continuous 3-min and 0.6Hz horizontal sinusoidal oscillation of the body support base. We hypothesized an ongoing temporal organization of the balancing strategy that gradually becomes fine-tuned and more coordinated with the platform movement. The trial was divided into oscillation cycles and for each cycle: leg muscles activity and temporal relationship between Centre of Mass and Centre of Pressure A-P position were analyzed. The results of each cycle were grouped in time-windows of 10 successive cycles (time windows of 16.6s). Muscle activity was initially prominent and diminished progressively. The major burst of Tibialis Anterior (TA) muscle always occurred at the same time instant of the platform oscillation cycle, in advance with respect to the platform posterior turning point. This burst produced a body forward rotation that was delayed throughout the task. During prolonged and repeatable balance perturbation, an ongoing postural adaptation process occurs. When the effects of the perturbation become predictable, the CNS scales the level of muscle activity to counteracting the destabilizing effects of the perturbations. Furthermore, the CNS tunes the kinematics and the kinetic responses optimally by slightly delaying the onset of the body forward rotation, maintaining unchanged the time-pattern of postural muscle activation.

Adaptation to continuous perturbation of balance: progressive reduction of postural muscle activity with invariant or increasing oscillations of the center of mass depending on perturbation frequency and vision conditions.

We investigated the adaptation of balancing behavior during a continuous, predictable perturbation of stance consisting of 3-min backward and forward horizontal sinusoidal oscillations of the support base. Two visual conditions (eyes-open, EO; eyes-closed, EC) and two oscillation frequencies (LF, 0.2 Hz; HF, 0.6 Hz) were used. Center of Mass (CoM) and Center of Pressure (CoP) oscillations and EMG of Soleus (Sol) and Tibialis Anterior (TA) were recorded. The time course of each variable was estimated through an exponential model. An adaptation index allowed comparison of the degree of adaptation of different variables. Muscle activity pattern was initially prominent under the more challenging conditions (HF, EC and EO; LF, EC) and diminished progressively to reach a steady state. At HF, the behavior of CoM and CoP was almost invariant. The time-constant of EMG adaptation was shorter for TA than for Sol. With EC, the adaptation index showed a larger decay in the TA than Sol activity at the end of the balancing trial, pointing to a different role of the two muscles in the adaptation process. At LF, CoM and CoP oscillations increased during the balancing trial to match the platform translations. This occurred regardless of the different EMG patterns under EO and EC. Contrary to CoM and CoP, the adaptation of the muscle activities had a similar time-course at both HF and LF, in spite of the two frequencies implying a different number of oscillation cycles. During adaptation, under critical balancing conditions (HF), postural muscle activity is tuned to that sufficient for keeping CoM within narrow limits. On the contrary, at LF, when vision permits, a similar decreasing pattern of muscle activity parallels a progressive increase in CoM oscillation amplitude, and the adaptive balancing behavior shifts from the initially reactive behavior to one of passive riding the platform. Adaptive balance control would rely on on-line computation of risk of falling and sensory inflow, while minimizing balance challenge and muscle effort. The results from this study contribute to the understanding of plasticity of the balance control mechanisms under posture-challenging conditions.

Equilibrium during static and dynamic tasks in blind subjects: no evidence of cross-modal plasticity.

Can visual information be replaced by other sensory information in the control of static and dynamic equilibrium? We investigated the balancing behaviour of acquired and congenitally blind subjects (25 severe visually impaired subjects--15 males and 10 females, mean age 36 +/- 13.5 SD) and age and gender-matched normal subjects under static and dynamic conditions. During quiet stance, the centre of foot pressure displacement was recorded and body sway analysed. Under dynamic conditions, subjects rode a platform continuously moving in the antero-posterior direction, with eyes open (EO) and closed (EC). Balance was inferred by the movement of markers fixed on malleolus, hip and head. Amplitude of oscillation and cross-correlation between body segment movements were computed. During stance, in normal subjects body sway was larger EC than EO. In blind subjects, sway was similar under both visual conditions, in turn similar to normal subjects EC. Under dynamic conditions, in normal subjects head and hip were partially stabilized in space EO but translated as much as the platform EC. In blind subjects head and hip displacements were similar in the EO and the EC condition; with respect to normal subjects EC, body displacement was significantly larger with a stronger coupling between segments. Under both static and dynamic conditions, acquired and congenitally blind subjects had a similar behaviour. We conclude that long-term absence of visual information cannot be substituted by other sensory inputs. These results are at variance with the notion of compensatory cross-modal plasticity in blind subjects and strengthen the hypothesis that vision plays an obligatory role in the processing and integration of other sensory inputs for the selection of the balancing strategy in the control of equilibrium.

Stance- and locomotion-dependent processing of vibration-induced proprioceptive inflow from multiple muscles in humans.

We performed a whole-body mapping study of the effect of unilateral muscle vibration, eliciting spindle Ia firing, on the control of standing and walking in humans. During quiet stance, vibration applied to various muscles of the trunk-neck system and of the lower limb elicited a significant tilt in whole body postural orientation. The direction of vibration-induced postural tilt was consistent with a response compensatory for the illusory lengthening of the stimulated muscles. During walking, trunk-neck muscle vibration induced ample deviations of the locomotor trajectory toward the side opposite to the stimulation site. In contrast, no significant modifications of the locomotor trajectory could be detected when vibrating various muscles of the lower as well as upper limb. The absence of correlation between the effects of muscle vibration during walking and standing dismisses the possibility that vibration-induced postural changes can account for the observed deviations of the locomotor trajectory during walking. We conclude that the dissimilar effects of trunk-neck and lower limb muscle vibration during walking and standing reflect a general sensory-motor plan, whereby muscle Ia input is processed according to both the performed task and the body segment from which the sensory inflow arises.

Trunk muscle proprioceptive input assists steering of locomotion.

During locomotion, human subjects navigate in their environment and choose the direction by means of the internal representation of space that is continuously updated by sensory input. Aim of this study was to assess whether trunk proprioceptive information plays a role in the definition of the reference frame for orientation. Unilateral trunk muscle vibration was applied during locomotion along a straight path in seven subjects. Vibration was administered either from the onset or in the middle of a seven-step task, under eyes-open (EO) or blindfolded condition. The deviation of the walking trajectory was quantified by the distance of the seventh from the first foot print along the medio-lateral axis. Foot angles and stride lengths were computed for all foot-falls. Vibration produced a clear-cut deviation from the straight-ahead direction when delivered in the middle of blindfolded locomotion. With EO the deviation was much smaller. A mild deviation was obtained in blindfolded condition when vibration started at the onset of locomotion. All deviations from the straight-ahead were accompanied by coherent changes in foot orientation on the ground. Trunk proprioception plays a major role in the definition of locomotor trajectory. Trunk input seems to be weighted against vision and whole-body kinematic information.

Neck muscle fatigue and spatial orientation during stepping in place in humans.

Neck proprioceptive input, as elicited by muscle vibration, can produce destabilizing effects on stance and locomotion. Neck muscle fatigue produces destabilizing effects on stance, too. Our aim was to assess whether neck muscle fatigue can also perturb the orientation in space during a walking task. Direction and amplitude of the path covered during stepping in place were measured in 10 blindfolded subjects, who performed five 30-s stepping trials before and after a 5-min period of isometric dorsal neck muscle contraction against a load. Neck muscle electromyogram amplitude and median frequency during the head extensor effort were used to compute a fatigue index. Head and body kinematics were recorded by an optoelectronic system, and stepping cadence was measured by sensorized insoles. Before the contraction period, subjects normally stepped on the spot or drifted forward. After contraction, some subjects reproduced the same behavior, whereas others reduced their forward progression or even stepped backward. The former subjects showed minimal signs of fatigue and the latter ones marked signs of fatigue, as quantified by the dorsal neck electromyogram index. Head position and cadence were unaffected in either group of subjects. We argue that the abnormal fatigue-induced afferent input originating in the receptors transducing the neck muscle metabolic state can modulate the egocentric spatial reference frame. Notably, the effects of neck muscle fatigue on orientation are opposite to those produced by neck proprioception. The neck represents a complex source of inputs capable of modifying our orientation in space during a locomotor task.

A new hip-knee-ankle-foot sling: kinematic comparison with a traditional ankle-foot orthosis.

In this study, we performed a kinematic analysis of a new, low-cost sling for the lower limb, compared to a common ankle-foot orthosis (AFO). Gait with no orthosis, with the AFO, and with the new sling was analyzed in one hemiplegic subject. Both the AFO and the sling reduced the mean angle and ROM (range of movement) of the ankle and the vertical displacement of the center of mass. The sling, but not the AFO, restored the normal sequence heel-strike, forefoot contact of the affected side. The sling, but not the AFO, reduced the affected limb stance and stride duration, increased stride length, and improved walking speed. In conclusion, the proposed sling for the lower limb equally improved the affected ankle kinematics in contrast to the traditional AFO, and it also improved some gait variables in this hemiplegic subject.